Papers by Josef van Genabith

Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation (2020.emnlp-main)

Challenge:	Language Technologies (LTs) are a powerful means to break down language barriers impacting business, cross-lingual and cross-cultural communication in Europe.
Approach:	They present an overview of the European LT landscape and the current state of play in industry and the LT market.
Outcome:	The present study outlines funding programmes, activities, actions and challenges in the different countries with regard to LT, including the current state of play in industry and the LT market.

Copied to clipboard

Dana Ruiter, Josef van Genabith, Cristina España-Bonet

Self-Supervised Neural Machine Translation (P19-1)

Challenge:	Existing studies on curriculum learning focus on selecting the best distribution of data to train a system.
Approach:	They propose a self-supervised neural machine translation model that self-selects data without being told to do so.
Outcome:	The proposed model self-selects samples of increasing complexity and task relevance without being told to do so, and performs a denoising curriculum.

Copied to clipboard

Dana Ruiter, Cristina España-Bonet, Josef van Genabith

Text Generation

Challenge:	Neural machine translation (NMT) methods relied on the availability of high-quality parallel corpora.
Approach:	They propose a method where an emergent NMT system is used for selecting training data and learning internal NMT representations.
Outcome:	The proposed method achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training.

Analysing Coreference in Transformer Outputs (D19-65)

Copied to clipboard

Ekaterina Lapshinova-Koltunski, Cristina España-Bonet, Josef van Genabith

Translation Quality Estimation by Jointly Learning to Score and Rank (2020.emnlp-main)

Challenge:	Using a transformer architecture, we study coreference phenomena in three neural machine translation systems.
Approach:	They analyse coreference phenomena in three neural machine translation systems . they manually annotate (the possibly incorrect) coreference chains in the outputs .
Outcome:	The proposed model shows stronger translationese effects in machine translated outputs than in human translations.

Copied to clipboard

Jingyi Zhang, Josef van Genabith

TransIns: Document Translation with Markup Reinsertion (2021.emnlp-demo)

Challenge:	The translation quality estimation (QE) task aims to evaluate the general quality of a translation without using reference translations.
Approach:	They propose a translation quality estimation task that uses translations as reference . they propose supervised learning using cross-lingual sentence embeddings from pre-trained multilingual models.
Outcome:	The proposed model outperforms sentBLEU on the WMT 2019 QE as a Metric task and outperformed sentBLUE on the QE in a multilingual language task.

Copied to clipboard

Jörg Steffen, Josef van Genabith

Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification (2021.emnlp-main)

Challenge:	MT models cannot translate complex formatted documents, as markup can be nested, apply to spans contiguous in source but non-contiguous.
Approach:	They propose a system for non-plain text document translation that reinserts markup into translated sentences using token alignments between source and target sentences.
Outcome:	The proposed system outperforms translation services in terms of markup quality . it integrates token alignments between source and target sentences to reinsert markup . the proposed system is available under the MIT license .

Copied to clipboard

Daria Pylypenko, Kwabena Amponsah-Kaakyire, Koel Dutta Chowdhury, Josef van Genabith, Cristina España-Bonet

Lipschitz Constrained Parameter Initialization for Deep Transformers (2020.acl-main)

Challenge:	Traditional hand-crafted features have been used for distinguishing between translated and original non-translated texts.
Approach:	They compare a feature-engineering-based approach to a features-learning-based one and use pre-trained neural word embeddings to train neural architectures.
Outcome:	The proposed approach outperforms other approaches by more than 20 accuracy points and the BERT-based model performs the best in both monolingual and multilingual settings.

Copied to clipboard

Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Jingyi Zhang

Semantic Text Processing

Challenge:	Existing studies show that deep Transformers have difficulty in training even with residual connection and layer normalization.
Approach:	They propose a method that leverages the Lipschitz constraint on the initialization of Transformer parameters to ease the optimization difficulties caused by its multi-layer encoder/decoder structure.
Outcome:	The proposed model outperforms previous RNN/CNN models but fails to converge with the original computation order.

Learning Hard Retrieval Decoder Attention for Transformers (2021.findings-emnlp)

Copied to clipboard

Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong

Find-2-Find: Multitask Learning for Anaphora Resolution and Object Localization (2023.emnlp-main)

Challenge:	In this paper, we show that learning a hard retrieval attention that attends to a single token in a sentence is 1.43 times faster than the standard scaled dot-product attention.
Approach:	They propose a method to learn hard retrieval attention where an attention head attends to a single token in a sentence rather than all tokens.
Outcome:	The proposed method is 1.43 times faster in decoding while preserving translation quality on a wide range of MT tasks.

Copied to clipboard

Cennet Oguz, Pascal Denis, Emmanuel Vincent, Simon Ostermann, Josef van Genabith

Information Extraction & Text Mining

Challenge:	Existing systems require large number of accurate annotations, such as image-level labels and location-level labeling.
Approach:	They propose a joint anaphora resolution and object localization dataset targeting visual-linguistic ambiguity.
Outcome:	The proposed framework improves visual-linguistic alignment and object localization with one joint model compared to a strong single-task baseline.

Learning Source Phrase Representations for Neural Machine Translation (2020.acl-main)

Copied to clipboard

Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu, Jingyi Zhang

Semantic Text Processing

Challenge:	Existing approaches to machine translation have been shown to be effective for long sentences . however, the attentional network can't capture long-distance dependencies .
Approach:	They propose a multi-head attention mechanism which generates phrase representations from token representations and incorporates them into the Transformer translation model to enhance its ability to capture long-distance relationships.
Outcome:	The proposed model can be computed in parallel and improves on the WMT 14 tasks.

MMPE: A Multi-Modal Interface for Post-Editing Machine Translation (2020.acl-main)

Copied to clipboard

Nico Herbig, Tim Düwel, Santanu Pal, Kalliopi Meladaki, Mahsa Monshizadeh, Antonio Krüger, Josef van Genabith

Multimodality

Challenge:	Current advances in machine translation (MT) increase the need for translators to switch from traditional translation to post-editing (PE) of machine-translated text.
Approach:	They propose to combine traditional input modes with pen, touch, and speech modalities for post-editing of machine-translated text.
Outcome:	The proposed interfaces are designed to reduce errors and save time.

Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change (2020.acl-main)

Copied to clipboard

Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu

Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers (2021.naacl-main)

Challenge:	Compared to previous studies, the performance of neural models is likely to be affected by the choice of hyper-parameters.
Approach:	They propose to automatically and dynamically determine batch sizes by accumulating gradients of mini-batches and performing an optimization step at just the time when the direction of gradients starts to fluctuate.
Outcome:	The proposed approach improves the Transformer model with a fixed 25k batch size by +0.73 and +0.82 BLEU respectively.

Copied to clipboard

Hongfei Xu, Josef van Genabith, Qiuhui Liu, Deyi Xiong

Mid-Air Hand Gestures for Post-Editing of Machine Translation (2021.acl-long)

Challenge:	Neural Machine Translation (NMT) has attracted wide attention in recent years.
Approach:	They propose a probing-based approach to measure word translation accuracy using transformer layers.
Outcome:	The proposed model outperforms previous probing-based translation models.

Copied to clipboard

Rashad Albo Jamara, Nico Herbig, Antonio Krüger, Josef van Genabith

Translating away Translationese without Parallel Data (2023.emnlp-main)

Challenge:	In a well-connected world, translation is of everincreasing importance.
Approach:	They propose to use mid-air hand gestures in combination with the keyboard for editing in machine translation and post-editing workflows to improve quality.
Outcome:	The proposed prototype supports mid-air hand gestures for cursor placement, text selection, deletion, and reordering.

Copied to clipboard

Rricha Jalota, Koel Chowdhury, Cristina España-Bonet, Josef van Genabith

Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)

Challenge:	Translated texts exhibit systematic linguistic differences compared to original texts in the same language, referred to as translationese . studies show translationeses have effects on various cross-lingual natural language processing tasks .
Approach:	They propose a translation-based style transfer approach that learns from monolingual data . they combine a self-supervised approach with an unsupervised approach .
Outcome:	The proposed method reduces translationese to a level of a random classifier after style transfer while preserving the content and fluency in the target original style.

Copied to clipboard

Sonal Sannigrahi, Josef van Genabith, Cristina España-Bonet

Representation Learning

Challenge:	obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data.
Approach:	They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models.
Outcome:	The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit.

Investigating the Helpfulness of Word-Level Quality Estimation for Post-Editing Machine Translation Output (2021.emnlp-main)

Copied to clipboard

Raksha Shenoy, Nico Herbig, Antonio Krüger, Josef van Genabith

A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment (2021.acl-long)

Challenge:	Post-editing (PE) machine translation (MT) output can save time and reduce errors.
Approach:	They propose to use automatic word-level quality estimation to predict correctness of MT output to flag problematic output.
Outcome:	The proposed model is not good enough to support human translations, but is based on a visualization reflecting uncertainty of the model.

Copied to clipboard

Jingyi Zhang, Josef van Genabith

The Transference Architecture for Automatic Post-Editing (2020.coling-main)

Challenge:	Existing methods for learning word alignment include statistical word aligners (e.g. GIZA++) Existing word alignment models employ a target-to-source attention mechanism which can provide rough word alignments but with a low accuracy.
Approach:	They propose a bidirectional Transformer based alignment model for unsupervised learning of the word alignment task.
Outcome:	The proposed model outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++ on three word alignment tasks.

Copied to clipboard

Santanu Pal, Hongfei Xu, Nico Herbig, Sudip Kumar Naskar, Antonio Krüger, Josef van Genabith

Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation (2021.acl-short)

Challenge:	A research challenge is the search for architectures that best support the capture, preparation and provision of src and mt information and its integration with pe decisions.
Approach:	They propose a multi-encoder based neural APE model that conditions post-editing decisions on both the source and machine translated text as inputs.
Outcome:	The proposed model outperforms the best performing systems by 1 BLEU point on the WMT 2016, 2017, and 2018 English–German APE shared tasks.

Copied to clipboard

Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong

Language Data Sharing in European Public Services – Overcoming Obstacles and Creating Sustainable Data Sharing Infrastructures (2020.lrec-1)

Challenge:	Existing work has increased the modeling capacity of multilingual NMT by deepening or widening the Transformer.
Approach:	They propose to increase the model capacity by deepening the Transformer . they propose to use a multi-input-multi-output architecture to combine multiple inputs .
Outcome:	The proposed model surpasses previous work and is 1.31 times faster than existing models.

Copied to clipboard

Lilli Smal, Andrea Lösch, Josef van Genabith, Maria Giagkou, Thierry Declerck, Stephan Busemann

Responsible & Trustworthy NLP

Challenge:	Data is key in training modern language technologies.
Approach:	They summarise findings of first pan-European study on barriers to language data sharing . they identify structural challenges, disposition towards CAT tools and lack of digital skills . overcoming language barriers is one of the main challenges european citizens face .
Outcome:	The paper summarises the findings of the first pan-European study on barriers to language data sharing . the findings highlight the barriers and recommend solutions to overcome them .

Understanding Translationese in Multi-view Embedding Spaces (2020.coling-main)

Copied to clipboard

Koel Dutta Chowdhury, Cristina España-Bonet, Josef van Genabith

Rewiring the Transformer with Depth-Wise LSTMs (2024.lrec-main)

Challenge:	Recent studies show footprints of the source language remain visible in translations . this is the first time departures from isomorphism between embedding spaces are used to track translationese.
Approach:	They exploit departures from isomorphism between spaces built from original target language and translations into this target language to predict relations between languages in an unsupervised way.
Outcome:	The proposed method exploits departures from isomorphism between embedding spaces to predict relations between languages in an unsupervised way.

Copied to clipboard

Hongfei Xu, Yang Song, Qiuhui Liu, Josef van Genabith, Deyi Xiong

When Your Cousin Has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages (2024.lrec-main)

Challenge:	Stacking non-linear layers allows deep neural networks to model complicated functions . but residual connections within each layer fail to fuse information from previous layers effectively .
Approach:	They propose a Transformer with depth-wise LSTMs connecting cascading Transformer layers and sub-layers.
Outcome:	The proposed model improves in English-German / French and multilingual tasks with BLEU.

Copied to clipboard

Niyati Bafna, Cristina España-Bonet, Josef van Genabith, Benoît Sagot, Rachel Bawden

Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation (2021.acl-long)

Challenge:	Existing methods for unsupervised bilingual lexicon induction depend on good quality static or contextual embeddings for both languages.
Approach:	They propose a method for unsupervised bilingual lexicon induction between a related LRL and a high-resource language that only requires inference on a masked language model of the HRL.
Outcome:	The proposed method performs well on low-resource languages with 5M tokens against Hindi . it is compared with existing methods on (mid-resourced) Marathi and Nepali .

Copied to clipboard

Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Meng Zhang

European Language Resource Coordination: Collecting Language Resources for Public Sector Multilingual Information Management (L18-1)

Challenge:	a self-attention network can be easily parallelized at sequence level, but LSTMs are slower to train . a recent study shows that LS models require a lot of computations to perform .
Approach:	They propose to compute LSTMs at sequence level to enable sequence-level parallelization . they use a bag-of-words representation of the preceding tokens context to approximate LStms .
Outcome:	The proposed model performs better than existing models while being faster to train . the model can be trained efficiently due to the highly parallelized self-attention network .

Copied to clipboard

Andrea Lösch, Valérie Mapelli, Stelios Piperidis, Andrejs Vasiļjevs, Lilli Smal, Thierry Declerck, Eileen Schnur, Khalid Choukri, Josef van Genabith

Exploring Paracrawl for Document-level Neural Machine Translation (2023.eacl-main)

Challenge:	European Language Resource Coordination (ELRC) initiated a number of actions to support the collection of Language Resources (LRs) within the public sector in EU member and CEF-affiliated countries.
Approach:	They propose to initiate actions to support the collection of Language Resources (LRs) within the public sector in EU member and CEF-affiliated countries.
Outcome:	The European Language Resource Coordination (ELRC) consortium initiated a number of actions to support the collection of Language Resources (LRs) within the public sector in EU member and CEF-affiliated countries.

Copied to clipboard

Yusser Al Ghussin, Jingyi Zhang, Josef van Genabith